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Abstract 

The  productivity  paradox  is  a  theory  that  suggests  that  investments  in  Information 
Technology  (IT)  do  not  necessarily  lead  to  associated  gains  in  the  productivity  of  the 
organization  (Malakoff,  2000;  Hitt  and  Brynjolfsson,  1996).  This  perception  leads 
practitioners  to  question  if  acquiring  new  IT  systems  for  the  sake  of  having  the  latest 
technology  will  make  their  organization  any  more  productive  (Liebmann,  1996). 
Understanding  the  problem  that  is  facing  the  practitioners,  this  research  was  undertaken 
to  attempt  to  answer  some  of  the  underlying  questions  relating  to  the  perceptions  held 
about  the  relationship  between  IT  expenditures  and  workplace  productivity  with  respect 
to  Air  Force  communication  squadrons. 

The  research  indicates  that  there  may  in  fact  be  a  perception  of  an  IT  productivity 
paradox.  Both  commanders  and  maintainers  feel  that  procurement  and  administrative 
changes  have  been  made  in  IT  planning  due  to  the  understanding  of  a  potential  IT 
productivity  paradox. 

Ultimately,  the  Air  Force  work  centers  have  the  perception  that  they  are  getting 
and  adequate  return  on  investment  for  IT  expenditures,  indicating  that  their  IT  planning 
procedures  have  been  effective.  However,  the  results  also  indicate  that  they  have  a 
perceived  need  for  newer  technologies  to  be  able  to  keep  their  network  infrastructures  to 
the  necessary  level  to  support  their  customer’s  needs.  This  indicates  that  IT  planning  in 
the  Air  Force  must  continually  change  to  strike  the  appropriate  balance  between  the 
demands  of  the  customers  and  the  capabilities  of  the  technologies. 
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THE  EFFECTS  OF  TECHNOLOGY  TURNOVER  ON  WORKPLACE 


PRODUCTIVITY  PERCEPTIONS 

I.  Introduction 

The  productivity  paradox  is  a  theory  that  suggests  that  investments  in  Information 
Technology  (IT)  do  not  lead  to  associated  gains  in  the  productivity  of  the  organization 
(Malakoff,  2000;  Hitt  and  Brynjolfsson,  1996).  Consequently,  since  its  first  postulation 
by  Nobel  Prize  winning  economist  Robert  Solow,  the  productivity  paradox  has  been  a 
serious  issue  that  has  confounded  scholars  and  practitioners  alike  (Malakoff,  2000). 
Academicians  debate  whether  this  phenomenon  is  real  or  just  a  perceived  anomaly  due  to 
the  mismeasurement  of  the  constructs  of  workplace  productivity  (Brynjolfsson,  1993), 
while  practitioners  are  just  beginning  realize  that  they  have  yet  to  find  a  positive 
correlation  between  the  considerable  sum  of  money  they  are  investing  in  information 
technology  (IT)  and  its  effects  on  workplace  productivity  (Liebmann,  1996). 
Compounding  this  already  perplexing  problem,  recent  studies  indicate  that  the  academic 
community’s  overall  perception  of  the  paradox  is  on  the  decline  (Hitt  and  Brynjolfsson, 
1996),  while  new  findings  suggest  that  product  development  life  cycles  are  decreasing 
and  thus  increasing  perceived  technology  obsolescence  rates  at  the  practitioner’s  level 
(Liebmann,  1996;  Peters,  2000;  Sichel,  1999). 

Ultimately,  the  uncertainty  brought  on  by  this  phenomena  indicates  that 
practitioners  are  starting  to  question  if  acquiring  new  IT  systems  for  the  sake  of  having 
the  latest  technology  makes  their  organization  any  more  productive  (Liebmann,  1996). 
Understanding  the  problem  that  is  facing  both  communities,  this  research  was  undertaken 


to  attempt  to  answer  some  of  the  underlying  questions  relating  to  the  relationship 
between  IT  expenditures  and  workplace  productivity  with  respect  to  Air  Force 
communication  squadrons. 

It  is  theorized  that  an  understanding  of  technology  turnover  may  have  increased 
the  practitioners’  perception  of  the  paradox  and  its  negative  effects  on  their  normal 
business  operations.  This  epiphany  may  be  responsible  for  a  trend  towards  a  more 
intelligent  IT  procurement  process  and  use  of  infonnation  resources  to  reduce  these 
perceived  impacts  on  IT  return  on  investment  (ROI). 

The  intention  of  this  research  is  not  to  look  for  evidence  of  the  existence  of  the 
productivity  paradox;  rather,  it  is  to  investigate  the  factors  that  influence  IT  planning  at 
the  workforce  component  and  whether  IT  expenditures  themselves  are  seen  as 
contributors  to  workplace  productivity.  Accordingly,  the  foundation  of  this  research 
focuses  on  the  impacts  of  technology  turnover  and  decreased  system  life  cycles  on  the 
perceptions  of  return  on  investment  (ROI)  for  IT  expenditures  of  both  the  senior 
leadership  and  network  administrators  in  the  Air  Force  community.  It  further  seeks  to 
identify  any  new  IT  acquisition  methodologies  that  have  been  initiated  to  realize  and 
measure  an  acceptable  level  of  ROI  from  the  practitioner’s  viewpoint. 

In  order  to  posit  a  relationship  between  IT  expenditures,  technology  turnover,  and 
workplace  productivity,  this  research  seeks  the  answers  to  five  general  questions  through 
the  use  of  a  53  question  7-point  Likert  scale  survey: 

1 .  Do  the  practitioner’s  have  a  general  perception  of  a  productivity  paradox? 

2.  Have  work  centers’  IT  planning  initiatives  been  influenced  by  the 
perception  of  a  productivity  paradox? 
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3.  What  are  the  current  justifications  for  IT  procurements  at  the  work  center 
level? 

4.  How  does  the  organization  view  IT  in  respect  to  workplace  productivity? 

5.  Is  the  senior  leadership’s  view  of  the  relationship  between  IT  investments 
and  productivity  significantly  different  from  their  network  administrators? 

With  sponsorship  from  the  Air  Force’s  Chief  Information  Office  (CIO),  these  five 
questions,  each  of  which  correspond  to  a  hypothesis  to  be  tested,  are  addressed  in  a 
survey  sent  to  all  Air  Force  Communications  Squadron  commanders  and  their  respective 
network  administrators. 

The  following  chapters  will  illustrate  specifically  how  this  research  was  pursued. 
The  next  chapter  is  the  literature  review  of  the  existing  knowledge  of  the  productivity 
paradox  and  other  foundational  studies  that  enabled  the  development  of  the  theories 
tested  in  this  research.  The  third  chapter  is  the  methodology  that  was  implemented  to 
collect,  analyze,  and  report  the  findings  of  the  research.  The  fourth  chapter  depicts  the 
results  and  analysis  used  to  measure  the  tests  of  the  research  hypotheses.  The  last  chapter 
provides  an  assessment  of  the  results  and  their  implications  to  the  Air  Force  as  a  result  of 
the  research. 

There  are  three  appendices  at  the  end  of  this  study  that  are  included  to  clarify  the 
analysis  of  the  results.  Appendix  A  shows  the  histograms  of  each  of  the  factors  relating 
to  their  associated  hypotheses  tests.  Appendix  B  illustrates  the  analysis  of  variance 
(ANOVA)  tests  that  were  performed  to  assess  Hypothesis  5.  Finally,  Appendix  C  depicts 
the  unequal  variance  tests  that  were  also  performed  to  assess  Hypothesis  5. 
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II.  Literature  Review 


Productivity  Paradox 

In  order  to  establish  the  baseline  for  this  research,  a  formal  understanding  of  the 
productivity  paradox  is  needed.  The  productivity  paradox  is  a  theory  that  suggests  that 
investments  in  Infonnation  Technology  (IT)  do  not  lead  to  associated  gains  in  the 
productivity  of  the  organization  (Malakoff,  2000;  Hitt  and  Brynjolfsson,  1996). 

However,  some  scholars  in  the  academic  community  rationalize  the  perception  of  the 
productivity  paradox  simply  as  a  confounding  problem  in  detennining  the  value,  in  terms 
of  productivity,  or  increase  in  output  generated,  by  IT  investments  (Brynjolfsson,  1993; 
Chan,  2000;  Due,  1994;  Mahmood  and  Mann,  1993).  More  specifically,  in  the  context  of 
this  research,  the  perception  of  the  productivity  paradox  will  be  described  in  accordance 
with  the  research  undertaken  by  Erik  Brynjolfsson  (Brynjolfsson,  1993),  see  Figure  1 
below. 


Figure  1:  Brynjolfsson’s  Theory  of  the  Productivity  Paradox 
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Brynjolfsson’s  study  (Brynjolfsson,  1993)  ultimately  concludes  that  there  are 
four  factors  that  contribute  to  this  perception  of  the  productivity  paradox.  They  are: 
mismeasurement  of  inputs  and  outputs,  lags  due  to  learning  and  adjustment, 
redistribution  and  dissipation  of  profits,  and  mismanagement  of  information  and 
technology  (p.  73). 

To  clarify  the  meanings  of  these  constructs  it  is  necessary  to  show  how 
Brynjolfsson  justifies  them.  He  starts  by  justifying  his  category  of  mismeasurement  of 
inputs  and  outputs  by  explaining  that,  “traditional  measures  of  the  relationship  between 
inputs  and  outputs  fail  to  account  for  non-traditional  sources  of  value.”  (pg  73) 
Furthermore,  he  illustrates  the  meaning  of  lags  due  to  learning  and  adjustment  by  stating 
that,  “if  significant  lags  between  cost  and  benefits  exist,  the  short-term  results  look  poor 
but  ultimately  the  payoff  will  be  proportionately  larger.”  (pg  73)  Next,  he  classifies  the 
third  construct  by  explaining  that  the,  “redistribution  argument  suggests  that  those 
investing  in  technology,  benefit  privately  but  at  the  expense  of  others,  so  no  net  benefits 
show  up  at  the  aggregate  level.”  (pg  73)  Finally,  he  explains  that  mismanagement  results 
from,  “something  in  IT’s  nature  that  leads  firms  or  industries  to  invest  in  it  when  they 
should  not,  to  misallocate  it,  or  to  use  it  to  create  slack  instead  of  productivity.”  (pg  73) 
(Brynjolfsson,  1993) 

Brynjolfsson  is  not  alone  in  his  description  of  the  problem.  Discussing 
Brynjolfsson’s  work,  (Chan,  2000)  states  that  the  productivity  paradox  of  infonnation 
technology  is  defined  as  “an  apparent  IT  investment  paradox  with  respect  to  economy¬ 
wide  productivity,  the  productivity  of  IT  capital  in  manufacturing,  and  the  productivity  of 
IT  capital  in  services”  (p.  226).  Furthermore,  according  to  (Brynjolfsson,  1993), 
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“computing  power  in  the  U.S.  economy  has  increased  by  more  than  two  orders  of 
magnitude  since  1970  yet  productivity,  especially  in  the  service  sector,  seems  to  have 
stagnated”  (p.  67).  These  issues  raise  many  eyebrows  in  the  business  community  as 
managers  attempt  to  establish  business  plans  and  budgets  for  product  lines.  The  obvious 
question  arises  in  the  mind  of  the  business  managers  as  to  how  much  capital  should  be 
invested  in  IT  and  what  are  the  impacts,  if  any,  to  productivity  that  can  be  expected  from 
this  investment  (Liebmann,  1996;  Mueller,  1997)?  Unfortunately,  in  today’s  business 
climate,  the  more  relevant  question  seems  to  be  how  long  can  I  expect  the  return  on 
information  technology  (IT)  investment  to  last  before  more  capital  is  needed  to  keep 
those  systems  fully  operational  (Liebmann,  1996)?  Subsequently,  this  enigma  of 
uncertainty  in  IT  investments  has  been  referred  to  as  the  productivity  paradox  of  IT 
investment  (Malakoff,  2000). 

Importantly,  not  all  researchers  agree  with  Brynjolfsson’s  analysis.  Others  have 
stepped  forward  to  challenge  his  theory.  (Due,  1994)  referencing  Paul  Strasmann,  the 
former  CIO  at  the  Department  of  Defense,  states  that  Brynjolfsson’s  study  was 
“fundamentally  flawed  because  it  fails  to  take  into  account  the  work  force  component  of 
IT  investments  that  can  account  for  as  much  as  90%  of  the  total  IT  spending”  (p.  76). 
This  spending  at  the  workforce  level  brings  into  question  the  acquisition  practices  of  the 
units  and  the  emerging  question  of  technology  turnover. 

Since  the  productivity  paradox  was  first  postulated  (1987),  its  mere  existence  has 
been  the  subject  of  many  debates  (Malakoff,  2000).  There  are  two  taxonomies  in  the 
academic  community  that  attempt  to  explain  the  paradox’s  existence.  One  side  of  the 
argument  contends  the  paradox  exists  and  IT  productivity,  whether  positively  or 
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negatively  affected,  can  be  measured  (Brynjolfsson,  1993).  The  other  side  dismisses  the 
paradox  as  perception  error  due  to  inadequate  measures  and  constructs  used  to  quantify 
and  define  it  (Chan,  2000). 

Recently,  research  has  indicated  that  the  perception  of  the  paradox  is  on  the 
decline  (Hitt  and  Brynjolfsson,  1996).  Hitt  and  Brynjolfsson  indicated  in  their  1996 
study  that  “  IT  has  increased  productivity  and  created  substantial  value  for  consumers.” 
(pg  121)  They  conclude  by  stating  that,  “there  is  no  inherent  contradiction  between 
increased  productivity,  increased  consumer  value,  and  unchanged  business  profitability.” 
(Hitt  and  Brynjolfsson,  1996)  (pg  121) 

However,  some  new  evidence  suggests  that  as  product  development  life  cycles 
decreased  and  thus  increased  technology  turnover  rates,  users  have  become  more  aware 
that  simply  acquiring  new  technology  for  its  own  sake  did  not  necessarily  make  their 
organization  more  productive  (Cavill,  2000;  Goyal,  2001;  Liebmann,  1996;  McHale, 
1999;  Sichell,  1999). 

Taken  by  themselves,  these  explanations  of  the  paradox  are  perplexing  enough, 
but  the  misconception  of  the  paradox  is  further  bolstered  by  certain  sects  of  the  academic 
community  that  employ  methodologies  that  fail  to  incorporate  the  totality  of  constructs 
and  measurement  devices  currently  available  to  assess  its  impacts  (Chan,  2000). 
Numerous  examples  of  this  mismeasurement  of  productivity  can  be  seen  in  the  research 
performed  by  Chan.  As  an  example,  (Chan,  2000)  reveals  that  most  research  on  the 
subject  has  focused  on  a  single  type  of  data  at  a  single  level  of  an  organization  to  draw 
conclusions  on  the  causes  and  perceptions  of  the  paradox  (p.  227). 
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Research  Direction 


It  is  the  contention  of  this  research  that  technology  turnover  is  the  primary 
contributing  factor  to  the  perception  of  the  paradox  from  the  workforce  component. 
Accordingly,  a  new  theory  began  to  take  shape  using  Brynjolfsson’s  (1993)  study  as  a 
baseline  and  Due’s  (1994)  criticism  of  that  study  as  an  initiator  for  a  further  investigation 
into  the  explanation  of  the  paradox.  Specifically,  the  criticism  that  Brynjolfsson’s  study 
fails  to  account  for  contributions  from  the  work  force  level  (Chan,  2000).  This  critical 
omission  is  the  cornerstone  of  the  research  undertaken  in  this  thesis  effort. 

It  is  necessary  to  clarify  the  meaning  of  Brynjolfsson’s  constructs  to  assure  that 
technology  turnover  is  not  already  incorporated  into  any  one  of  them.  He  states  that,  “the 
first  two  explanations  point  to  shortcomings  in  research,  not  practice,  as  the  root  of  the 
productivity  paradox”  (p.  73).  He  continues  by  justifying  his  category  of 
mismeasurement  of  inputs  and  outputs  by  explaining  that,  “traditional  measures  of  the 
relationship  between  inputs  and  outputs  fail  to  account  for  non-traditional  sources  of 
value”  (p.  73).  Furthennore,  he  illustrates  the  meaning  of  lags  due  to  learning  and 
adjustment  by  stating  that,  “if  significant  lags  between  cost  and  benefits  exist,  the  short¬ 
term  results  look  poor  but  ultimately  the  payoff  will  be  proportionately  larger”  (p.  73). 
Next,  he  classifies  the  third  construct  by  explaining  that  the,  “redistribution  argument 
suggests  that  those  investing  in  technology,  benefit  privately  but  at  the  expense  of  others, 
so  no  net  benefits  show  up  at  the  aggregate  level”  (p.  73).  Finally,  he  explains  that 
mismanagement  results  from,  “something  in  IT’s  nature  that  leads  firms  or  industries  to 
invest  in  it  when  they  should  not,  to  misallocate  it,  or  to  use  it  to  create  slack  instead  of 
productivity”  (Brynjolfsson,  1993)  (p.  73).  Considering  the  analysis  of  these 
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justifications,  it  is  easily  seen  that  while  these  categories  address  both  the  shortcomings 
of  research  methods  and  the  difficulties  with  quantifying  productivity,  they  do  not 
address  technology  turnover  specifically.  Therefore,  it  is  proposed  that  this  research  is 
novel  to  the  community  and  worthy  of  further  investigation. 

The  theory  being  tested  here  indicates  that  technology  turnover,  based  on 
shortened  systems  development  life  cycles,  has  a  correlation  to  the  practitioner 
community’s  (in  this  case  the  Air  Force)  perception  of  an  IT  paradox  and  could  aid  in  the 
explanation  of  the  perceived  lack  of  productivity  at  the  work  center  level.  See  Figure  2 
below: 
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Figure  2:  Hypotheses  and  constructs  tested  in  this  research 
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In  addition  to  suggesting  a  possible  further  explanation,  this  research  seeks  to 
determine  how  this  perception  impacts  work  center  productivity  and  return  on  investment 
(ROI).  In  doing  so,  this  research  investigates  the  IT  planning  strategies  of  Air  Force 
work  centers  to  see  what  is  driving  their  IT  procurements.  The  IT  planning  strategies  and 
procurements  element  was  added  with  five  inputs  tested  to  see  what  their  perceived 
influences  may  be.  As  shown  above,  the  inputs  to  this  planning  process  are:  perceptions 
of  the  paradox,  technology  turnover  itself,  availability  of  funds,  customer  satisfaction, 
and  environmental  forces.  These  are  the  five  factors  that  were  chosen  to  be  tested  and 
should  not  be  viewed  as  an  all  inclusive  list  of  contributing  factors.  They  were  chosen 
specifically  to  seek  the  answers  to  the  fundamental  questions  of  this  research  as  outline 
previously.  Finally,  a  series  of  questions  were  developed  to  determine  the  impacts  the 
current  Air  Force  work  center  IT  strategies  have  on  the  perceptions  of  productivity  and 
ROI. 

It  is  important  to  understand  that  the  primary  focus  of  this  research  is  not  to 
debate  the  existence  of  the  paradox,  but  rather  to  simply  investigate  the  perceptions  held 
by  the  Air  Force  IT  community  as  to  the  effects  that  IT  investments  have  had  on 
workplace  productivity.  More  specifically,  this  research  seeks  to  demonstrate  that  recent 
trends  yielding  reduced  systems  development  life  cycles  for  computer  and  IT 
technologies  have  impacted  these  perceptions  in  such  a  way  as  to  force  a  shift  in  IT 
systems  acquisition  thinking  (Cavill,  2000;  Goyal,  2001;  Liebmann,  1996;  McHale,  1999; 
Sichell,  1999).  These  factors  may  in  fact  support  the  purported  existence  of  the  paradox 
and  a  more  rigorous  explanation  of  it,  but  this  is  an  outcome  of  the  research  that  should 
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be  studied  in  greater  detail  on  a  larger  scale  before  any  causal  relationships  can  be 
established. 

The  main  objective  of  this  research  is  to  see  how  technology  turnover,  through 
reduced  IT  system  development  life  cycles,  impacts  the  perception  of  current  network 
viability  at  Air  Force  organizations.  Additionally,  this  study  gauges  how  the  Air  Force 
community  views  the  utility  of  IT  technologies  and  detennine  the  reasons  for  new 
purchases  and  the  productivity  perceptions  they  have  on  these  new  systems.  A  final  goal 
of  the  research  was  to  detennine  if  there  is  a  perceived  technology  need  gap  between  Air 
Force  senior  leaders  and  network  administrators. 

The  objectives  of  this  research  are  linked  to  the  five  general  questions  that  were 
outlined  previously.  These  questions  have  been  transformed  into  hypotheses  to  be  tested 
by  this  empirical  study  and  are  listed  and  below: 

Hypothesis  1  (HI):  The  understanding  of  technology  turnover  at  the  work  center  level 
of  the  Air  Force  has  led  to  a  perception  of  an  IT  productivity  paradox. 

Hypothesis  2  (H2):  The  perception  of  the  paradox  has  positively  influenced  the  IT 
planning  strategies  at  the  work  center  level  in  the  Air  Force. 

Hypothesis  3a  (H3a):  The  understanding  of  technology  turnover  has  positively 
impacted  Air  Force  IT  procurements  process  at  the  work  center  level. 

Hypothesis  3b  (H3b):  Availability  of  funding  has  negatively  impacted  Air  Force  IT 
procurements  process  at  the  work  center  level. 

Hypothesis  3c  (H3c):  Environmental  factors,  such  as  customers,  mission,  and  higher 
headquarter  direction,  have  negatively  impacted  Air  Force  IT  planning  at  the  work  center 
level. 
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Hypothesis  3d  (H3d):  Customer  satisfaction  positively  influences  IT  planning  at  the 
work  center  level. 

Hypothesis  4  (H4):  IT  procurements  are  not  perceived  as  contributors  to  work  center 
productivity. 

Hypothesis  5  (H5):  The  senior  leadership  at  the  work  center  level  has  a  significantly 
different  level  of  understanding  of  technology  turnover  and  its  effects  on  IT  planning 
than  their  network  maintainers. 

The  ideas  contained  in  this  research  effort  were  briefed  to  senior  personnel  at  the 
Air  Force  CIO  office  and  they  eagerly  sponsored  this  effort.  They  were  active  in  the 
construction  of  the  survey  instrument  and  provided  a  list  of  all  Air  Force  Communication 
Squadron  commanders  as  the  potential  respondents  to  this  survey. 

The  results  of  this  research  could  have  a  great  impact  to  the  current  IT 
procurement  methodologies  employed  throughout  the  Air  Force.  However,  the  research 
is  not  aimed  to  assign  blame  to  or  applaud  any  organizations,  rather  it  is  to  be  used  as  a 
litmus  test  for  the  senior  leaders  to  see  where  the  Air  Force  is  in  terms  of  dealing  with  the 
problems  of  IT  expenditures  and  work  center  productivity. 
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ITT.  Methodology 


Survey  Selection 

In  order  to  test  the  constructs  of  the  research  undertaken  a  7-point  Likert  scale 
format  for  the  questions  was  chosen.  The  Likert  scale  was  chosen  as  the  intention  of  the 
survey  is  to  measure  the  perception  and  tendencies  of  the  respondents  to  questions  about 
work  center  productivity  and  technology  turnover.  The  Likert  scale  was  chosen  for  its 
simplicity  and  ability  to  obtain  inputs  on  perceptions. 

A  7-point  scale  was  chosen  to  allow  for  maximum  scaling  of  the  respondents 
perceptions  to  the  questions  asked.  The  scale  was  constructed  as  follows:  1  =  Strongly 
Disagree,  2  =  Disagree,  3  =  Slightly  Disagree,  4  =  Neutral,  5  =  Slightly  Agree,  6  =  Agree, 
and  7  =  Strongly  Agree.  It  is  felt  that  a  Likert  scale  of  a  5-point  nature  would  limit  and 
bunch  the  responses  around  the  median  response  of  3.  A  7-point  scale  allows  for  a 
response  farther  away  from  the  median  response  without  having  to  select  one  of  the 
extremes  unless  the  respondent  feels  absolutely  compelled  to.  Additionally,  the  6-point 
Likert  scale  was  eliminated  from  consideration  as  it  was  desired  to  permit  a  neutral 
response  to  the  questions  to  allow  for  a  possible  elimination  of  questions  that  may  not 
affect  perceptions  either  positively  or  negatively.  Therefore  a  search  for  an  existing 
Likert  scale  survey  instrument  was  undertaken. 

Due  to  the  specific  ties  to  technology  turnover  and  its  applicability  to  return  on 
investment,  a  detailed  search  of  existing  survey  instruments  was  performed  by  accessing 
the  ISWorld  website  (http://www.isworld.org).  It  was  found  that  no  specific  survey 
instrument  addressed  all  the  factors  that  were  desired  to  be  tested.  Certain  concepts  were 
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noted  for  possible  use  in  construction  of  a  new  survey  at  a  later  date,  but  no  single  survey 
was  available  for  use  in  this  research  effort.  This  necessitated  the  construction  of  a 
unique  survey  instrument  to  meet  the  needs  of  this  research. 

Survey  Construction 

After  inspection  of  the  available  survey  instruments,  it  was  decided  that  the  best 
way  to  test  the  constructs  of  the  research  was  to  develop  a  new  instrument.  Two 
reference  books  on  survey  construction  were  reviewed  (Kalton,  1990;  Converse  and 
Presser,  1990)  and  a  list  of  34  questions  was  developed  to  address  the  five  main  areas  of 
research.  This  list  of  questions  was  reviewed  and  approved  by  the  research  advisor  and 
sponsor  of  this  research  and  a  set  of  obverse  questions  were  developed  to  allow  for  a 
reliability  check  for  each  of  the  34  questions.  This  list  of  68  questions  was  then 
randomly  sorted  in  an  attempt  to  remove  any  ordering  bias  that  might  be  introduced. 

This  final  list  of  questions  was  then  sent  to  the  Air  Force  Personnel  Center  (AFPC)  for  a 
formal  review  of  the  questions  and  approval  for  dissemination. 

Survey  Approval 

The  survey  was  sent  to  AFPC  for  approval  as  it  was  to  appear  on  the  Air  Force 
Institute  of  Technology  (AFIT)  hosted  website.  Initial  AFPC  response  was  that  the 
survey  was  too  long  and  the  reliability  questions  made  it  seem  overly  redundant.  It  was 
suggested  that  1 5  of  the  obverse  questions  be  removed  bringing  the  total  number  of 
questions  to  53.  Therefore,  the  reliability  questions  were  segregated  into  the  five  primary 
focus  areas.  It  was  desired  that  any  one  of  these  areas  not  be  left  without  a  form  of 
reliability  check  to  the  questions  therefore  it  was  decided  that  a  specific  percentage  from 
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each  section  would  be  removed  so  as  not  to  leave  any  one  area  lacking  reliability 
questions.  Each  section  was  considered  to  be  independent  of  the  others  and  reliability 
questions  were  removed  at  random  from  each  section  until  the  selected  removal 
percentage  was  achieved.  In  total  15  reliability  questions  were  removed  at  random 
without  leaving  any  one  of  the  main  focus  areas  without  a  check  for  reliability.  The  53 
question  survey  was  then  revised  and  sent  to  AFPC  for  their  review  and  the  survey  was 
approved  and  given  an  official  Air  Force  survey  control  number,  USAF  SCN  02-096,  that 
expired  on  31  Dec  2002. 

Pilot  Study 

The  pilot  study  was  undertaken  in  two  parts.  The  first  part  of  the  study  was  a 
functionality  check  to  see  that  the  survey  was  readable  and  the  collection  mechanism  was 
sufficiently  gathering  and  storing  the  responses.  A  group  of  2 1  students  were  chosen 
from  the  Information  Resource  Management  program  at  AFIT.  This  group  was  selected 
for  their  familiarity  of  the  subject  and  the  assurance  of  timely  and  meaningful  feedback  of 
survey  access  and  data  collection  performance.  Questions  were  revised  and  structural 
changes  were  made  to  the  web-based  survey  for  aesthetic  reasons  based  on  the  responses 
received  from  this  initial  pilot  study  group. 

The  second  part  of  the  pilot  study  involved  sending  the  survey  to  an  actual  Air 
Force  communication  squadron  for  their  response.  Information  was  given  as  to  the 
anonymity  of  their  responses  and  the  desire  for  survey  feedback.  The  initial  feedback 
from  the  commander  of  this  communications  squadron  led  to  a  reevaluation  of  the  target 
audience  of  the  survey.  The  particular  communications  squadron  surveyed  had  recently 
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outsourced  most  of  its  IT  related  activities  to  a  contract  organization  with  government 
civilian  oversight.  The  initial  pool  of  respondents  was  to  be  military  only,  so  a  revision 
was  made  to  the  survey  to  allow  for  the  commanders  to  redistribute  the  survey  to  the 
appropriate  management  official.  The  commanders  were  informed  that  the  survey  could 
only  be  completed  by  civilian  or  contractor  personnel  on  a  strictly  voluntary  basis.  It  is 
understood  that  the  Air  Force  is  moving  towards  outsourcing  IT  activities  and  this  may 
have  been  a  limiting  factor  in  the  number  of  respondents  that  participated  in  the  research. 
The  pilot  study  was  then  sent  to  the  designated  civilian  authority  for  the  chosen 
communications  squadron,  but  no  responses  were  ever  received.  In  the  interest  of  time, 
the  pilot  study  was  ended  at  this  point.  Indications  were  that  this  could  be  a  tenninal 
problem  with  the  receipt  of  responses  for  the  survey,  but  only  if  the  majority  of  the 
communications  squadrons  to  be  surveyed  had  also  been  outsourced.  At  this  point  the 
decision  was  made  to  send  out  the  survey  to  the  masses  to  see  if  this  was  going  to  limit 
the  respondents  to  an  unacceptable  number.  If  this  would  have  been  the  case,  a  major 
adjustment  to  the  collection  of  the  data  would  have  to  be  made  and  this  early  notice 
would  have  left  time  to  make  those  changes.  Ultimately,  this  was  not  a  terminal  factor  in 
the  data  collection  methodology  and  the  research  was  able  to  continue. 

Survey  Dissemination 

The  population  chosen  to  respond  to  this  survey  is  of  utmost  significance  to  the 
Air  Force.  The  Air  Force  has  119  communication  squadrons  and  their  commanders  are 
solely  responsible  for  managing  and  implementing  the  Air  Force’s  multi-billion  dollar 
communications  budget  and  associated  information  technology  resources.  Of  these  119 
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commanders,  only  112  email  addresses  could  be  found  for  inclusion  in  the  dissemination 
of  the  survey.  The  survey  was  sent  to  these  1 12  Air  Force  communication  squadron 
commanders,  but  4  of  the  emails  were  rejected  because  of  outdated  email  addresses.  This 
meant  that  the  sample  size  of  the  population  to  be  survey  was  108.  Of  these  108 
commanders  two  of  the  respondents  sent  back  justifications  as  to  why  this  survey  was  not 
appropriate  for  their  organizations  due  to  the  very  specific  missions  of  these 
organizations.  It  was  agreed  that  these  two  organizations  were  outside  the  original  scope 
of  this  research  so  their  responses  were  not  sought.  This  meant  that  the  final  number  of 
respondents  intended  to  be  surveyed  was  106.  These  106  commanders  or  their 
representatives  were  then  asked  to  take  the  survey  themselves  and  also  to  disseminate  the 
survey  to  their  associated  network  maintainers  for  their  response. 

Response  Retrieval 

This  survey  was  hosted  on  AFIT’s  computer  network  and  the  hyperlink  to  this 
web  page  was  sent  to  each  of  the  respondents  to  ease  their  access  to  the  survey.  The  web 
page  was  developed  to  allow  for  automatic  collection  of  the  survey  data  in  a  database, 
once  the  respondent  submitted  the  answers.  The  respondents  were  asked  to  provide  any 
further  comments  they  had  at  the  end  of  the  survey  and  provided  a  means  to  request  a 
final  version  of  this  research  if  desired. 

Analysis  Methods 

Once  the  surveys  were  completed  and  the  data  collected,  analysis  of  the  data  was 
performed.  The  first  step  was  to  perform  a  factor  analysis  of  the  individual  hypotheses’ 
responses  to  make  sure  that  the  series  of  questions  in  fact  loaded  onto  the  factor  that  was 
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intended.  Any  additional  factors  that  were  found  were  recorded  and  the  model  changed 
to  reflect  their  inclusion.  Additionally,  the  hypotheses  were  treated  as  separate 
independent  events  to  simplify  and  remove  any  cross  loading  effects  between  them.  The 
following  is  a  complete  list  of  the  original  questions  for  each  of  the  hypotheses.  The 
revised  list  to  include  the  segregation  of  questions  into  distinct  factors  is  included  in  the 
next  chapter  of  this  report.  In  the  list  of  questions  that  follow,  the  “Neg”  in  parentheses 
indicate  questions  that  would  have  to  be  negatively  scored  to  allow  for  proper  assessment 
of  an  overall  score  of  a  factor  as  these  were  the  obverse  questions  that  were  generated  for 
reliability  checks  as  mentioned  previously. 

Hypothesis  1  Questions 

4.  Your  organization  achieves  an  adequate  return-on-investment  (ROI)  for  its  IT 
expenditures  before  the  system  has  to  be  upgraded. 

5.  Recent  technological  advances  are  beginning  to  render  your  current  IT 
infrastructure  obsolete. 

8.  Recent  IT  advances  have  required  your  organization  to  procure  new  systems  to 
maintain  the  current  level  of  mission  readiness  and  connectivity. 

35(Neg).  Your  organization  fails  to  achieve  an  adequate  return-on-investment 
(ROI)  from  previous  IT  investments  prior  to  seeking  upgrades  for  your  current 
infrastructure. 

47.  Your  organization  was  able  to  maximize  the  potential  of  your  current  IT 
infrastructure  before  you  made  your  latest  major  IT  purchase. 
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Hypothesis  2  Questions 

l(Neg).  Your  organization  does  NOT  keep  metrics  on  IT  investments  or  their 
impacts  on  the  user/customer  (such  as  satisfaction,  productivity,  or  resource 
savings). 

9.  The  network  administrators  in  your  organization  keep  current  with  emerging 
technologies  in  the  IT  industry. 

10.  The  network  administrators  or  managers  in  your  organization  are  empowered 
to  research  new  IT  technologies  in  advance  of  any  need  for  network  upgrades. 
14(Neg).  The  recent  rate  of  technological  advances  in  the  computer  industry  has 
NOT  affected  your  organization’s  IT  procurement  processes. 

19(Neg).  Your  organization  fails  to  formally  review  the  new  technology  need 
against  the  current  systems  capabilities  prior  to  initiating  the  procurement  process. 
24.  Your  organization  maintains  metrics  on  IT  investments  and  their  impacts  on 
the  user/customer  (such  as  satisfaction,  productivity,  or  resource  savings). 

3  l(Neg).  The  network  administrators  or  managers  in  your  organization  are  limited 
in  their  ability  to  research  new  IT  technologies  in  advance  of  any  need  for 
network  upgrades. 

33.  The  recent  rate  of  technological  advances  in  the  computer  industry  has  led  to  a 
refinement  in  your  organization’s  IT  procurement  processes. 

37.  The  increasing  rates  of  technological  advancement  in  the  IT  industry  have 
increased  your  organization’s  awareness  and  planning  for  network  system’s 
procurement. 
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42(Neg).  Your  organization’s  awareness  and  planning  for  network  system’s 
procurement  has  NOT  been  affected  by  the  increasing  rates  of  technological 
advancement  in  the  IT  industry. 

49.  Your  organization  formally  reviews  the  new  technology  need  against  current 
system  capabilities  prior  to  initiating  the  procurement  process. 

5 1 .  Before  making  your  latest  IT  purchase,  your  organization  perfonned  a 
detailed  analysis  of  your  current  system’s  capabilities. 

53.  The  persons  responsible  for  IT  procurements  in  your  organization  meet  with 
network  administrators  and  managers  about  emerging  technologies  on  a  regular 
basis. 

Hypothesis  3a  Questions 

18.  Your  organization  must  maintain  currency  in  the  state-of  the-art  technological 
advances  to  continue  to  perform  its  mission. 

23.  Your  organization  avoids  procuring  new  computer  hardware  and/or  software 
technologies  simply  because  they  are  state-of-the-art  and  available. 

29(Neg).  State-of-the-art  technological  advances  are  NOT  required  for  your 
organization  to  continue  to  perform  its  mission. 

34(Neg).  Your  organization’s  computer  hardware  and/or  software  purchases  are 
based  availability  of  new  technologies. 

45.  Your  organization’s  computer  hardware  and/or  software  purchases  are  NOT 
based  strictly  on  availability  of  new  technologies. 
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Hypothesis  3b  Questions 

3(Neg).  A  primary  source  for  funding  innovative  computer  hardware  and/or 
software  technologies  in  your  organization  is  through  end  of  year  funds  or 
“fallout”  money. 

11.  Your  organization’s  computer  hardware  and/or  software  purchases  are  based 
on  available  funding. 

12.  IT  funding  is  a  major  part  of  your  organization’s  budget. 

16(Neg).  IT  funding  is  a  relatively  small  part  of  your  organization’s  budget. 

22.  The  funding  for  IT  procurements  and/or  maintenance  in  your  organization  is 
adequate  to  sustain  the  mission  requirements  of  the  users. 

28.  The  primary  source  for  funding  innovative  computer  hardware  and/or 
software  technologies  in  your  organization  is  through  the  normal  budget  process 
and  NOT  through  the  use  of  “fallout”  funds. 

50.  Your  organization  assures  that  there  are  training  funds  in  the  annual  budget 
for  user  training  of  new  computer  system  hardware  and/or  software  components. 

Hypothesis  3c  Questions 

20(Neg).  Recent  upgrades  to  your  IT  infrastructure  have  been  based  on  higher 
headquarter  mandates  or  regulations. 

25.  The  leadership  in  your  organization  feels  that  IT  infrastructure  is  a  critical  part 
of  the  user’s  ability  to  meet  mission  requirements. 

26.  The  recent  upgrades  to  your  IT  infrastructure  were  NOT  downward  directed. 
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27(Neg).  The  leadership  in  your  organization  does  NOT  see  IT  infrastructure  as  a 
critical  part  of  the  user’s  ability  to  meet  mission  requirements. 

36.  Your  organization’s  users/customers  drive  the  requirements  for  new  IT 
procurements. 

38.  Your  organization’s  computer  hardware  and/or  software  purchases  are  based 
on  mission  need. 

46(Neg).  Your  organization’s  computer  hardware  and/or  software  purchases  are 
NOT  based  strictly  on  mission  need. 

Hypothesis  3d  Questions 

2.  Your  organization’s  users/customers  have  little  influence  in  the  procurement  of 
new  IT  technologies. 

6.  Your  organization’s  current  infrastructure  is  capable  of  meeting  the  user’s 
requirements  in  the  short  term  (1-2  yrs). 

39.  Your  organization’s  current  infrastructure  is  capable  of  meeting  the  user’s 
requirements  in  the  long  tenn  (2-5  years). 

43.  Your  organization’s  users/customers  are  satisfied  with  the  current  network 
infrastructure  and  capabilities. 

Hypothesis  4  Questions 

7(Neg).  Recent  IT  purchases  have  NOT  provided  your  users/customer  with  new 
products  and  services  to  enhance  mission  completion. 
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13(Neg).  Your  latest  major  IT  purchase  was  NOT  critical  to  your  system 
users ’/customers’  ability  to  continue  to  perfonn  their  mission  related  tasks. 
15(Neg).  Recent  IT  expenditures  have  NOT  permitted  your  users/customer’s  to 
expand  their  mission  capabilities. 

17.  Recent  IT  purchases  in  your  organization  have  been  made  to  increase 
user/customer  productivity. 

21.  Your  latest  IT  purchases  have  increased  your  network  personnel’s 
productivity  in  completing  their  mission. 

30.  Your  organization  has  been  able  to  expand  the  users’/customers’  mission 
capabilities  because  of  recent  IT  expenditures. 

32.  Your  organization  has  been  able  to  provide  new  products  and  services  to  the 
user/customers  to  enhance  mission  completion  because  of  recent  IT  purchases. 
40(Neg).  Your  latest  IT  purchases  have  NOT  affected  the  time  needed  by  the 
customers  to  complete  mission  related  tasks. 

41.  New  innovations  in  IT  have  enabled  your  organization  to  reallocate  network 
personnel  or  resources  to  enhance  mission  success. 

44(Neg).  The  new  innovations  in  IT  have  NOT  permitted  your  organization  to 
redistribute  network  personnel  to  enhance  mission  success. 

48.  Your  latest  IT  purchases  have  reduced  the  time  needed  by  the  users/customers 
to  complete  mission  related  tasks. 

52.  Your  latest  major  IT  purchase  was  critical  to  the  ability  of  your  system’s 
users/customers  to  continue  to  perfonn  mission  tasks. 
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After  the  factor  analysis  was  completed  and  the  model  revised,  the  results  of  the 
factors  were  computed  by  taking  the  mean  of  means  of  the  responses  to  the  individual 
questions  for  each  factor.  The  result  was  a  score  that  could  be  compared  to  the  Likert 
scale  factors  to  determine  if  the  respondents  agreed  with  the  hypothesis  or  disagreed. 

To  test  Hypothesis  5,  the  scores  of  the  distinct  factors  for  each  of  the  seven 
previous  hypotheses  of  the  two  groups  were  compared  through  an  analysis  of  variance 
(ANOVA).  This  was  performed  to  determine  if  there  was  a  statistically  significant 
difference  between  the  responses  received  by  the  commanders  and  maintainers  for  each 
of  the  factors.  Additionally,  a  test  of  unequal  variances  was  perfonned  between  the  two 
groups  for  each  of  the  hypothesis  factors  to  determine  if  the  total  number  of  respondents 
or  the  difference  in  number  of  respondents  per  group  would  impact  the  results  and 
implications. 

Summary 

This  chapter  provided  the  methodology  used  to  gather  information  on  the  level  of 
the  perception  of  an  IT  productivity  paradox  at  the  work  center  level  of  the  Air  Force.  A 
web-based  survey  was  used  to  gather  the  research  data.  The  subjects  for  this  study  were 
stratified  into  communication  squadron  commanders  and  network  maintainers.  An  email 
was  sent  to  the  all  Air  Force  communication  squadron  commanders  and  their  respective 
network  maintainers  to  inform  them  of  the  study  and  the  web  location  for  its  access.  Due 
to  its  relatively  small  size  the  entire  population  was  selected  for  this  research  effort. 

After  the  subjects  completed  the  survey,  the  results  were  computed  to  provide  a  means  of 
analysis.  The  next  chapter  provides  an  analysis  of  the  survey  responses. 
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IV.  Results 


Demographics 

The  total  number  of  respondents  to  the  survey  was  72.  However,  to  compute  the 
response  rate  it  is  necessary  to  refer  to  the  number  of  commanders  that  participated. 
Figure  3  illustrates  that  21  of  the  106  commanders  that  were  contacted  responded  to  the 
survey  or  had  their  designated  authority  respond  yielding  a  response  rate  of  19.81%. 


Commanders 

21 

29.17% 

Maintainers 

51 

70.83% 

Surveys  sent 

106 

Surveys  received 

21 

Response  rate 

19.81% 

Contractor 

2 

2.78% 

Civilian  Employee 

9 

12.50% 

General 

0 

0% 

Colonel 

0 

0% 

Lt  Colonel 

7 

9.72% 

Major 

8 

11.11% 

Captain 

11 

15.28% 

1  Lt 

3 

4.17% 

2Lt 

5 

6.94% 

CMSgt 

1 

1.39% 

SMSgt 

1 

1.39% 

MSgt 

5 

6.94% 

TSgt 

8 

11.11% 

SSgt 

5 

6.94% 

SrA 

3 

4.17% 

A1C 

4 

5.56% 

Amn 

0 

0% 

AB 

0 

0% 

0  - 1  yr 

14 

19.44% 

1  -  3  yrs 

12 

16.67% 

3  -  5  yrs 

8 

11.11% 

5  -  7  yrs 

7 

9.72% 

7  - 10  yrs 

8 

11.11% 

10  plus  yrs 

23 

31.94% 

33S 

34 

47.22% 

3C0 

16 

22.22% 

3C2 

7 

9.72% 

2E2 

3 

4.17% 

3A0 

1 

1.39% 

Contractor 

2 

2.78% 

Civilian 

9 

12.50% 

Figure  3:  Survey  Results  Demographics 
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Figure  3  also  shows  that  these  commanders  sent  the  survey  to  their  maintainers 
and  5 1  of  them  responded.  Due  to  the  size  of  each  of  the  groups,  2 1  and  5 1 ,  the  Central 
Limit  Theorem  can  be  invoked,  thus  making  an  assumption  of  normality  for  the  statistical 
distribution  of  responses  possible. 

Factor  Analysis  and  Results 

The  following  infonnation  is  the  factor  analysis  and  results  for  Hypotheses  1 
through  4.  The  factor  analysis  was  perfonned  using  both  standard  and  Verimax  rotated 
methodologies  to  provide  the  most  accurate  segregation  of  the  questions.  The  first  two 
tables  illustrate  the  how  the  hypothesis  questions  were  segregated  into  the  corresponding 
components.  The  questions  are  then  listed  under  the  names  of  the  new  factors  that  were 
created  as  a  result  of  this  factor  analysis.  The  last  table  under  each  hypothesis  shows  the 
actual  results  of  the  mean  of  means  calculations  and  corresponding  standard  deviations. 

A  revised  model  with  the  new  factors  included  is  shown  at  the  end  of  this  section  in 
Figure  4. 

Hypothesis  1 


Table  1:  Component  Matrix  for  test  of  Hypothesis  1 


Component 

1 

2 

Q47 

.784 

-.039 

Q35N 

.766 

.057 

04 

.742 

-.166 

05 

-.186 

.800 

Q8 

.376 

.687 
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Table  2  Rotated  Component  Matrix  for  test  of  Hypothesis  1 


Component 

1 

2 

Q47 

.784 

.026 

Q35N 

.759 

.120 

04 

1753 

-.104 

05 

-.252 

.782 

08 

.318 

1716 

Return  on  Investment 

47.  Your  organization  was  able  to  maximize  the  potential  of  your  current 
IT  infrastructure  before  you  made  your  latest  major  IT  purchase. 

35(Neg).  Your  organization  fails  to  achieve  an  adequate  retum-on- 
investment  (ROI)  from  previous  IT  investments  prior  to  seeking  upgrades 
for  your  current  infrastructure. 

4.  Your  organization  achieves  an  adequate  retum-on-investment  (ROI)  for 
its  IT  expenditures  before  the  system  has  to  be  upgraded. 


Technology  advancement 

5.  Recent  technological  advances  are  beginning  to  render  your  current  IT 
infrastructure  obsolete. 

8.  Recent  IT  advances  have  required  your  organization  to  procure  new 
systems  to  maintain  the  current  level  of  mission  readiness  and 
connectivity. 
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Table  3:  Results  of  test  of  Hypothesis  1 


Group 

Factor 

Mean 

Standard 

Deviation 

Commanders 

Retum-on-investment 

.6570 

5.1587 

1.5471 

Commanders 

Technological 

advances 

.0325 

5.0475 

1.5452 

Maintainers 

Retum-on-investment 

.2499 

4.5000 

1.6274 

Maintainers 

Technological 

advances 

.6895 

4.8300 

1.6009 

Hypothesis  2 


Table  4:  Component  Matrix  for  test  of  Hypothesis  2 


Component 

1 

2 

Q53 

.823 

-.003 

Q49 

.798 

.022 

Q51 

.786 

.027 

Q10 

.640 

-.465 

Q37 

.736 

-.092 

Q31N 

.619 

-.424 

Q19N 

.726 

.090 

Q24 

.704 

.171 

09 

.461 

-.446 

Q1N 

.611 

.218 

Q42N 

.214 

1707 

Q14N 

.179 

1710 

033 

.388 

.517 
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Table  5:  Rotated  Component  Matrix  for  test  of  Hypothesis  2 


Component 

1 

2 

Q53 

.792 

.223 

Q49 

.761 

.240 

051 

.749 

.242 

Q10 

.743 

-.271 

Q37 

.732 

.114 

Q31N 

.712 

-.238 

Q19N 

.674 

.286 

Q24 

.630 

.358 

09 

.566 

-.303 

Q1N 

.528 

.377 

Q42N 

.011 

.738 

Q14N 

- 

.023 

.732 

Q33 

.231 

.603 

Process  changes 

53.  The  persons  responsible  for  IT  procurements  in  your  organization 
meet  with  network  administrators  and  managers  about  emerging 
technologies  on  a  regular  basis. 

49.  Your  organization  formally  reviews  the  new  technology  need  against 
current  system  capabilities  prior  to  initiating  the  procurement  process. 

5 1 .  Before  making  your  latest  IT  purchase,  your  organization  perfonned  a 
detailed  analysis  of  your  current  system’s  capabilities. 

10.  The  network  administrators  or  managers  in  your  organization  are 
empowered  to  research  new  IT  technologies  in  advance  of  any  need  for 
network  upgrades. 
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37.  The  increasing  rates  of  technological  advancement  in  the  IT  industry 
have  increased  your  organization’s  awareness  and  planning  for  network 
system’s  procurement. 

31(Neg).  The  network  administrators  or  managers  in  your  organization  are 
limited  in  their  ability  to  research  new  IT  technologies  in  advance  of  any 
need  for  network  upgrades. 

19(Neg).  Your  organization  fails  to  formally  review  the  new  technology 
need  against  the  current  systems  capabilities  prior  to  initiating  the 
procurement  process. 

24.  Your  organization  maintains  metrics  on  IT  investments  and  their 
impacts  on  the  user/customer  (such  as  satisfaction,  productivity,  or 
resource  savings). 

9.  The  network  administrators  in  your  organization  keep  current  with 
emerging  technologies  in  the  IT  industry. 

l(Neg).  Your  organization  does  NOT  keep  metrics  on  IT  investments  or 
their  impacts  on  the  user/customer  (such  as  satisfaction,  productivity,  or 
resource  savings). 

Administrative  changes 

42(Neg).  Your  organization’s  awareness  and  planning  for  network 
system’s  procurement  has  NOT  been  affected  by  the  increasing  rates  of 
technological  advancement  in  the  IT  industry. 
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14(Neg).  The  recent  rate  of  technological  advances  in  the  computer 
industry  has  NOT  affected  your  organization’s  IT  procurement  processes. 
33.  The  recent  rate  of  technological  advances  in  the  computer  industry  has 
led  to  a  refinement  in  your  organization’s  IT  procurement  processes. 


Table  6:  Results  of  test  of  Hypothesis  2 


Group 

Factor 

mmmm 

Mean 

Standard 

Deviation 

Commanders 

Procurement  process 
changes 

.8405 

4.9571 

1.5782 

Commanders 

Administrative  process 
changes 

.5299 

4.6507 

1.7053 

Maintainers 

Procurement  process 
changes 

.8852 

4.2823 

1.8443 

Maintainers 

Administrative  process 
changes 

.6593 

4.4052 

1.5912 

Hypothesis  3a 


Table  7:  Component  Matrix  for  test  of  Hypothesis  3a 


Component 

1 

2 

Q45 

.745 

-.294 

023 

.711 

-.268 

Q34N 

.159 

-.547 

Q29N 

.256 

1775 

Q18 

.524 

.568 
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Table  8:  Rotated  Component  Matrix  for  test  of  Hypothesis  3a 


Component 

1 

2 

Q45 

.801 

.020 

Q23 

.759 

.031 

Q34N 

.359 

-.441 

Q29N 

-.066 

i.813 

Q18 

.261 

.727 

Availability  of  new  technologies 

45.  Your  organization’s  computer  hardware  and/or  software  purchases  are 
NOT  based  strictly  on  availability  of  new  technologies. 

23.  Your  organization  avoids  procuring  new  computer  hardware  and/or 
software  technologies  simply  because  they  are  state-of-the-art  and 
available. 


Perceived  need  of  new  technologies 

29(Neg).  State-of-the-art  technological  advances  are  NOT  required  for 
your  organization  to  continue  to  perform  its  mission. 

18.  Your  organization  must  maintain  currency  in  the  state-of  the-art 
technological  advances  to  continue  to  perform  its  mission. 
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Table  9:  Results  of  test  of  Hypothesis  3a 


Group 

Factor 

Mean 

Standard 

Deviation 

Commanders 

Technology  availability 

.5585 

4.9761 

1.6746 

Commanders 

Perceived  need  of  new 
technology 

.5007 

5.5476 

1.6260 

Maintainers 

Technology  availability 

.4344 

4.6862 

1.7404 

Maintainers 

Perceived  need  of  new 
technology 

.4503 

5.0588 

1.5721 

Hypothesis  3b 


Table  10:  Component  Matrix  for  test  of  Hypothesis  3b 


Component 

1 

2 

Q16N 

.822 

.420 

Q12 

.799 

.457 

Q50 

.683 

.157 

Q22 

.618 

-.312 

Q28 

.667 

-.520 

Q3N 

.510 

-.502 

oil 

- 

.033 

.679 

Table  11:  Rotated  Component  Matrix  for  test  of  Hypothesis  3b 


Component 

1 

2 

Q16N 

.917 

.110 

Q12 

.918 

.066 

Q50 

.655 

.250 

Q22 

.339 

j.604 

Q28 

.264 

-804 

Q3N 

.143 

^701 

Oil 

.351 

-.582 
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Budgeted  funding 

16(Neg).  IT  funding  is  a  relatively  small  part  of  your  organization’s 
budget. 

12.  IT  funding  is  a  major  part  of  your  organization’s  budget. 

50.  Your  organization  assures  that  there  are  training  funds  in  the  annual 
budget  for  user  training  of  new  computer  system  hardware  and/or  software 
components. 

Fallout  funding 

28.  The  primary  source  for  funding  innovative  computer  hardware  and/or 
software  technologies  in  your  organization  is  through  the  normal  budget 
process  and  NOT  through  the  use  of  “fallout”  funds. 

3(Neg).  A  primary  source  for  funding  innovative  computer  hardware 
and/or  software  technologies  in  your  organization  is  through  end  of  year 
funds  or  “fallout”  money. 

22.  The  funding  for  IT  procurements  and/or  maintenance  in  your 
organization  is  adequate  to  sustain  the  mission  requirements  of  the  users. 
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Table  12:  Results  of  test  of  Hypothesis  3b 


Group 

Factor 

Cronbach’s 

Alpha 

Mean 

Standard 

Deviation 

Commanders 

Availability  of 

Funds 

.7515 

4.5238 

2.1316 

Commanders 

Fallout  Funding 

.6779 

2.7301 

1.7616 

Maintainers 

Availability  of 

Funds 

.8470 

3.8169 

1.9915 

Maintainers 

Fallout  Funding 

.6757 

3.0588 

1.7999 

Hypothesis  3c 


Table  13:  Component  Matrix  for  test  of  Hypothesis  3c 


Component 

1 

2 

Q38 

.800 

.131 

Q25 

.781 

.117 

Q27N 

.750 

.109 

Q46N 

.703 

-.334 

036 

.413 

-.131 

Q20N 

- 

.040 

.926 

Q26 

.053 

.901 

Table  14:  Rotated  Component  Matrix  for  test  of  Hypothesis  3c 


Component 

1 

2 

Q38 

.800 

.131 

Q25 

.781 

.117 

Q27N 

.750 

.108 

Q46N 

.703 

-.334 

Q36 

.413 

-.132 

Q20N 

- 

.039 

.926 

Q26 

.053 

.901 
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Internal  forces 


38.  Your  organization’s  computer  hardware  and/or  software  purchases  are 
based  on  mission  need. 

25.  The  leadership  in  your  organization  feels  that  IT  infrastructure  is  a 
critical  part  of  the  user’s  ability  to  meet  mission  requirements. 

27(Neg).  The  leadership  in  your  organization  does  NOT  see  IT 
infrastructure  as  a  critical  part  of  the  user’s  ability  to  meet  mission 
requirements. 

46(Neg).  Your  organization’s  computer  hardware  and/or  software 
purchases  are  NOT  based  strictly  on  mission  need. 

36.  Your  organization’s  users/customers  drive  the  requirements  for  new  IT 
procurements. 

External  forces 

20(Neg).  Recent  upgrades  to  your  IT  infrastructure  have  been  based  on 
higher  headquarter  mandates  or  regulations. 

26.  The  recent  upgrades  to  your  IT  infrastructure  were  NOT  downward 
directed. 
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Table  15:  Results  of  test  of  Hypothesis  3c 


Group 

Factor 

Cronbach’s  Alpha 

Mean 

Standard  Deviation 

Commanders 

Internal  forces 

.7285 

5.5238 

1.3451 

Commanders 

External  forces 

.7735 

2.6190 

1.5294 

Maintainers 

Internal  forces 

.7076 

5.0431 

1.6583 

Maintainers 

External  forces 

.8418 

3.4117 

1.9106 

Hypothesis  3d 


Table  16:  Component  Matrix  for  test  of  Hypothesis  3d 


Component 

1 

2 

Q2N 

-.043 

1.999 

Q43 

.800 

-.001 

Q39 

.780 

.040 

06 

.780 

.016 

Table  17:  Rotated  Component  Matrix  for  test  of  Hypothesis  3d 


Component 

1 

2 

Q2N 

-.010 

1.000 

Q43 

.799 

-.028 

Q39 

.781 

.014 

06 

.780 

-.010 

Customer  satisfaction 

43.  Your  organization’s  users/customers  are  satisfied  with  the  current 
network  infrastructure  and  capabilities. 

39.  Your  organization’s  current  infrastructure  is  capable  of  meeting  the 
user’s  requirements  in  the  long  term  (2-5  years). 
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6.  Your  organization’s  current  infrastructure  is  capable  of  meeting  the 
user’s  requirements  in  the  short  tenn  (1-2  yrs). 


Table  18:  Results  of  test  of  Hypothesis  3d 


Group 

Factor 

Cronbach’s 

Alpha 

Mean 

Standard 

Deviation 

Commanders 

Customer 

satisfaction 

.7919 

4.5873 

1.8546 

Maintainers 

Customer 

satisfaction 

.6276 

4.4379 

1.6890 

Hypothesis  4 


Table  19:  Component  Matrix  for  test  of  Hypothesis  4 


Component 

1 

2 

Q21 

.781 

-.081 

030 

.773 

-.046 

Q15N 

.761 

.036 

Q17 

.743 

-.194 

Q48 

.724 

-.044 

Q52 

.708 

-.239 

032 

.657 

-.168 

Q13N 

.556 

-.434 

Q7N 

.545 

.080 

Q40N 

.538 

-.014 

Q44N 

.482 

.752 

Q41 

.515 

^718 
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Table  20:  Rotated  Component  Matrix  for  test  of  Hypothesis  4 


Component 

1 

2 

Q21 

.744 

.253 

Q30 

.722 

.282 

Q15N 

.677 

.351 

Q17 

.756 

.134 

Q48 

.676 

.262 

Q52 

.743 

.079 

Q32 

.667 

.121 

Q13N 

.686 

-.162 

Q7N 

.462 

.301 

Q40N 

.494 

.212 

Q44N 

.124 

.885 

Q41 

.168 

.867 

Mission  enhancement 

21.  Your  latest  IT  purchases  have  increased  your  network  personnel’s 
productivity  in  completing  their  mission. 

30.  Your  organization  has  been  able  to  expand  the  users  ’/customers’ 
mission  capabilities  because  of  recent  IT  expenditures. 

15(Neg).  Recent  IT  expenditures  have  NOT  permitted  your 
users/customers  to  expand  their  mission  capabilities. 

17.  Recent  IT  purchases  in  your  organization  have  been  made  to  increase 
user/customer  productivity. 

48.  Your  latest  IT  purchases  have  reduced  the  time  needed  by  the 
users/customers  to  complete  mission  related  tasks. 

52.  Your  latest  major  IT  purchase  was  critical  to  the  ability  of  your  system 
users ’/customers’  to  continue  to  perform  mission  tasks. 
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32.  Your  organization  has  been  able  to  provide  new  products  and  services 
to  the  users/customers  to  enhance  mission  completion  because  of  recent  IT 
purchases. 

13(Neg).  Your  latest  major  IT  purchase  was  NOT  critical  to  your  system 
users ’/customers’  ability  to  continue  to  perfonn  their  mission  related  tasks. 
7(Neg).  Recent  IT  purchases  have  NOT  provided  your  users/customers 
with  new  products  and  services  to  enhance  mission  completion. 

40(Neg).  Your  latest  IT  purchases  have  NOT  affected  the  time  needed  by 
the  customers  to  complete  mission  related  tasks. 

Reallocation  of  resources 

44(Neg).  The  new  innovations  in  IT  have  NOT  permitted  your 
organization  to  redistribute  network  personnel  to  enhance  mission  success. 
41.  New  innovations  in  IT  have  enabled  your  organization  to  reallocate 
network  personnel  or  resources  to  enhance  mission  success. 


Table  21:  Results  of  test  of  Hypothesis  4 


Group 

Factor 

Cronbach’s 

Alpha 

Mean 

Standard 

Deviation 

Commanders 

Mission  completion 
enhancement 

.9056 

5.2952 

1.3478 

Commanders 

Reallocation  of  resources 

.9256 

3.7380 

2.0959 

Maintainers 

Mission  completion 
enhancement 

.8513 

4.8823 

1.4065 

Maintainers 

Reallocation  of  resources 

.7226 

3.5882 

1.5244 
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New  Model 


Figure  4  illustrates  the  factors  that  make  up  the  hypotheses  that  were 
tested  in  this  research.  It  is  important  to  note  that  each  hypothesis  was  treated  as 
an  independent  event,  so  the  suggestion  of  causal  relationships  between 
hypotheses  is  not  an  intention  of  this  research. 
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Figure  4:  New  Thesis  Model 
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Deleted  Questions 

The  following  questions  were  deleted  due  to  reliability  issues  and  insufficient 
factor  loading  in  the  factor  analysis. 

Question  2  did  not  load  on  the  intended  factor  of  Hypothesis  3d  which  was 
customer  satisfaction.  Since  there  was  no  other  factor  that  was  intended  to  be  tested  and 
there  was  only  one  question  that  loaded  on  this  additional  factor,  this  question  was 
eliminated.  This  was  done  as  reliability  on  a  factor  with  only  one  question  was  difficult 
to  assess. 

Question  1 1  loaded  moderately  on  one  factor,  but  in  the  data  analysis  this 
question  had  severely  negative  effects  on  reliability  of  the  factor  it  loaded  on.  Reliability 
was  significantly  increased  with  the  omission  of  this  question  and  therefore  was  removed. 

Question  34  did  not  load  on  either  the  original  factor  or  the  new  factor  that  was 
revealed  after  the  factor  analysis.  Since  there  was  no  other  factor  that  was  intended  to  be 
tested  and  there  was  only  one  question  that  loaded  on  this  additional  factor,  this  question 
was  eliminated.  This  was  done  as  reliability  on  a  factor  with  only  one  question  was 
difficult  to  assess. 
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Hypothesis  5 

Table  22  shows  the  results  of  the  test  of  Hypothesis  5.  Included  in  the  table  are 
the  results  of  the  ANOVA  to  include  the  student’s  t  test,  the  Tukey-Kramer  means  test, 
and  the  Levene  test  of  unequal  variance.  An  alpha  of  .05  was  chosen  to  give  a  95% 
confidence  interval  of  the  ANOVA  results. 
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Table  22:  Results  of  test  of  Hypothesis  5 


Factor  tested 

Student’s  t 
(alpha  =.05) 

Tukey-Kramer 
HSD  (alpha  =.05) 

Unequal 

variances 

(Levene) 

Results 

HI  (Return-on- 
investment) 

.1676 

-.13918 

.4948 

No  significant 
difference 

HI  (Technological 
advances) 

.0647 

-.03377 

.2229 

No  significant 
difference 

H2  (Procurement 
process  changes) 

<  .0001 

.38971 

<.0001 

Significant 

difference 

H2  (Administrative 
changes) 

.3139 

-.23396 

.3469 

No  significant 
difference 

H3a  (Technology 
availability) 

.3699 

-.33409 

.6592 

No  significant 
difference 

H3a  (Perceived 
need  of  new 
technology) 

.0954 

-.08671 

.9477 

No  significant 
difference 

H3b  (Budgeted 
Funds) 

.0212 

.10691 

.1799 

Significant 

difference 

H3b  (Fallout 
Funding) 

.2211 

-.19920 

.5487 

No  significant 
difference 

H3c  (Internal 
forces) 

.0088 

.12179 

.0006 

Significant 

difference 

H3c  (External 
forces) 

.0181 

.13714 

.0129 

Significant 

difference 

H3d  (Customer 
satisfaction) 

.5666 

-.36364 

.2596 

No  significant 
difference 

H4  (Mission 

completion 

enhancement) 

.0003 

.18918 

.2337 

Significant 

difference 

H4  (Reallocation  of 
resources) 

.6332 

-.46960 

<.0001 

No  significant 
difference 

In  interpreting  the  results  of  Table  22,  it  should  be  known  that  a  Tukey-Kramer 
result  that  is  positive  yields  a  result  of  a  significant  difference  in  the  means.  This  can  also 
be  verified  by  the  fact  that  the  student  t-test  is  less  than  the  chosen  alpha  of  .05.  The 
Levene  unequal  variance  test  indicates  possible  problems  with  the  difference  in  the  size 
of  the  groups  when  the  result  is  less  than  the  given  alpha  of  .05.  This  would  indicate  that 
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caution  should  be  used  when  generalizing  the  results  to  the  entire  population.  Each  of 
these  situations  is  bolded  in  the  table  above  to  illustrate  it  when  they  occurred. 
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V.  Implications  to  the  Air  Force 


This  chapter  addresses  the  results  and  interpretations  of  the  research  as  well  as  the 
overall  implications  to  the  Air  Force.  Additionally,  the  chapter  identifies  the  limitations 
of  the  research  and  posits  possible  future  research  to  be  perfonned  to  further  the 
knowledge  in  this  area. 

Implication  to  Air  Force 

Each  hypothesis  is  repeated  below  followed  by  an  interpretation  of  the  results  and 
an  associated  implication  to  the  research. 

Hypothesis  1 

The  understanding  of  technology  turnover  at  the  work  center  level  of 
the  Air  Force  had  led  to  a  perception  of  an  IT  productivity  paradox. 

Interpretation 

Both  groups  feel  that  they  get  an  adequate  return  on  investment  for 
IT  related  purchases,  but  they  also  feel  that  they  need  to  continually 
upgrade  their  systems  due  to  technological  advances  and  user 
requirements. 
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Implications 


Due  to  the  low  alpha  of  the  technology  advancement  factor  of  the 
model  for  this  hypothesis,  one  cannot  generalize  this  particular  construct 
Air  Force-wide.  A  better  survey  instrument  is  needed  to  test  the 
technological  advancement  factor  to  allow  for  proper  measurement. 
However,  the  return-on-investment  factor  has  a  moderate  alpha  and  can  be 
generalized.  Therefore,  the  research  can  only  indicate  that  there  is  not  a 
great  perception  of  a  productivity  paradox  from  a  return-on-investment 
standpoint  at  the  Air  Force’s  work  center  level,  thus  Hypothesis  1  is  thus 
not  supported. 

Hypothesis  2 

The  perception  of  a  paradox  has  positively  influenced  the  IT  planning 
strategies  at  the  work  center  level  in  the  Air  Force. 

Interpretation 

Even  though  a  perception  of  a  paradox  cannot  be  supported 
(Hypothesis  1  results),  both  groups  feel  that  process  and  administrative 
changes  have  been  made  to  account  for  its  foundational  symptoms.  This 
apparent  disagreement  with  the  perception  of  the  paradox  could  be 
resolved  if  a  better  instrument  was  used  for  the  technology  advancements 
factor  of  Hypothesis  1.  This  factor  may  indeed  indicate  that  they  do  have 


47 


a  perception  of  the  paradox  that  has  led  to  the  changes  indicated  by  the 
results  of  this  hypothesis. 


Implications 

From  the  results  above  one  can  see  that  both  of  the  groups  feel  that 
a  positive  influence  on  IT  planning  has  taken  place  due  to  a  realization  of 
a  perceived  existence  of  an  IT  paradox.  This  is  contrary  to  the  findings  of 
Hypothesis  1 ,  but  this  could  be  explained  due  to  the  low  reliability  of  the 
technology  advancement  factor  of  the  model  for  Hypothesis  1 .  The 
process  changes  factor  has  a  high  alpha  and  the  administrative  changes 
factor  has  a  moderate  alpha,  but  the  process  changes  factor  has  an 
indication  of  a  possible  bias  due  to  unequal  variances.  Therefore,  caution 
should  be  used  when  making  a  generalization  to  the  entire  population  even 
though  the  Cronbach’s  alpha  for  each  factor  is  moderate  to  high. 
Considering  the  limitations  outlined  above,  Hypothesis  2  is  supported. 

Hypothesis  3a 

The  understanding  of  technology  turnover  has  positively  impacted 
Air  Force  IT  planning  at  the  work  center  level. 

Interpretation 

Both  groups  feel  that  availability  of  technologies  is  not  a  driving 
force  in  IT  planning.  However,  they  both  have  a  perceived  need  for  new 
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technologies  to  continue  to  meet  mission  and  customer  requirements. 
These  two  results  indicate  that  technology  turnover  is  impacting  IT 
planning,  but  not  due  to  the  mere  availability  of  new  technologies. 


Implications 

The  commander  group  alphas  are  moderate  and  the  maintainer 
group  alphas  are  low.  This  would  indicate  a  possible  problem  with  the 
survey  instrument.  A  misconception  or  misread  of  the  questions  is  evident 
between  the  two  groups.  This  results  in  a  measurement  of  this  hypothesis 
that  is  not  very  reliable  and  suspect  to  generalization.  Considering  these 
limitations,  Hypothesis  3a  is  only  partially  supported.  This  is  due  to  the 
low  alpha  of  the  maintainer’ s  responses  which  indicates  that  full  support 
to  the  hypothesis  cannot  be  made  at  this  time,  even  though  there  is  a 
perceived  need  of  new  technologies,  and  a  perception  that  the  mere 
availability  of  new  technologies  does  not  impact  IT  planning. 

Hypothesis  3b 

Availability  of  funding  has  negatively  impacted  Air  Force  IT  planning 
at  the  work  center  level. 

Interpretation 

The  two  groups  differ  on  their  perception  of  the  appropriate  level 
of  funding  for  IT  expenditures.  The  commanders  seem  to  indicate  that  the 
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current  level  of  budgeted  funding  is  adequate,  whereas  the  maintainers 
feel  that  the  current  level  is  not.  This  indicates  a  significant  disconnect  in 
the  perceptions  of  funding  ideologies  for  IT.  It  is  understood  that  the 
means  for  this  factor  are  both  close  to  the  median  of  four,  indicating  that 
the  perceptions  are  not  significantly  great  in  either  of  the  cases,  but  the 
difference  is  significant  enough  to  raise  a  question  for  future  studies  to 
investigate  this  potential  problem.  The  disconnect  between  the  two 
groups’  perceptions  could  lead  to  the  commanders  not  approving  IT 
initiatives  or  not  budgeting  for  them  even  if  the  maintainers  feel  that  the 
new  initiatives  are  warranted.  However,  both  groups  feel  that  the  majority 
of  IT  funding  is  through  end-of-year  or  fallout  money  rather  than  budgeted 
expenses. 

Implications 

Both  groups  have  moderate  to  high  alphas  for  this  model,  therefore 
the  factors  and  their  results  are  reliable  and  generalizable  to  the  Air  Force. 
Due  to  the  disconnect  between  perceptions  of  appropriate  IT  budget 
funding,  it  is  clear  that  this  factor  does  indeed  negatively  impact  IT 
planning  at  the  work  center  level.  The  maintainers  who  are  the  closest  to 
the  operational  aspects  of  the  networks  do  not  feel  adequate  funding  is 
available.  These  maintainers  are  relying  on  commanders  to  provide 
budgeted  funds  for  upgrades  to  these  systems,  and  these  commanders 
already  feel  that  the  funding  is  adequate.  This  could  make  procurement  of 
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new  IT  innovations  difficult.  Additionally,  these  results  indicate  that  the 
organizations  are  reliant  on  fallout  funds  to  make  IT  improvements. 

Again,  the  commanders  that  are  in  charge  of  the  distribution  of  these 
fallout  funds  already  feel  adequate  coverage  for  IT  funding  exists, 
potentially  exacerbating  the  problem.  Lastly,  by  the  nature  and 
availability  of  fallout  funds,  this  means  that  certainty  on  system  upgrades 
is  suspect,  negatively  impacting  IT  procurements.  Therefore  Hypothesis 
3b  is  supported. 

Hypothesis  3c 

Environmental  factors,  such  as  customers,  mission  and  higher 
headquarter  direction;  have  negatively  impacted  Air  Force  IT  planning  at 
the  work  center  level. 

Interpretation 

Both  groups  feel  that  internal  forces  such  as  mission  requirements, 
user  needs,  and  leadership  influences  have  a  positive  impact  on  IT 
procurements.  However,  both  groups  also  indicate  that  the  recent  IT 
expenditures  have  been  downward  directed,  resulting  in  a  negative  impact 
on  procurements. 
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Implications 


Both  groups  have  high  alphas,  but  there  are  indications  of  a 
possible  bias  due  to  unequal  variances.  Therefore,  caution  should  be  used 
when  making  a  generalization  to  the  entire  population  even  though  the 
Cronbach’s  alpha  for  each  factor  is  high.  The  results  indicate  that  the 
work  centers  are  required  to  coordinate  higher  headquarter  demands  with 
user  and  mission  needs  for  IT  expenditure  dollars.  These  results  indicate 
that  a  negative  impact  on  IT  procurements  exists  due  to  work  center 
environmental  factors,  thus  Hypothesis  3c  is  supported.  However,  it  is 
impossible  to  gauge  the  level  of  conflict  between  internal  and  external 
forces  with  this  model.  An  additional  study  into  this  phenomenon  is 
needed  to  show  the  actual  relationship  of  these  two  factors  and  the  impacts 
it  has  to  work  center  productivity.  The  results  here  simply  indicate  that  a 
conflict  between  internal  and  external  forces  exists. 


Hypothesis  3d 

Customer  satisfaction  positively  influences  IT  planning  at  the  work 
center  level. 

Interpretation 

Both  groups  feel  that  their  users  and/or  customers  are  satisfied  with 
the  current  infrastructure  and  its  ability  to  meet  their  short  and  long  tenn 
needs.  However,  this  model  fails  to  measure  or  gauge  how  this 
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satisfaction  relates  to  work  center  IT  planning.  This  results  from  a 
question  that  was  included  in  the  survey  being  improperly  constructed. 

The  question  was  removed  as  it  severely  impacted  the  reliability  of  the 
model.  A  new  survey  with  a  form  of  this  question  more  clearly  addressed 
is  needed  to  test  this  hypothesis. 

Implications 

Both  groups  have  to  moderate  to  high  alphas  indicating  that  this 
model  is  reliable  and  generalizable  to  the  Air  Force  forjudging  customer 
satisfaction.  However,  the  instrument  lacks  an  adequate  measure  of  how 
customer  satisfaction  influences  work  center  IT  planning,  due  to  the 
removal  of  Question  2  for  reliability  reasons.  Therefore,  this  instrument  is 
unable  to  measure  the  desired  response  to  Hypothesis  3d,  and  thus  cannot 
be  supported.  It  is  important  to  note  that  customer  satisfaction  is  an 
important  part  of  IT  planning  and  should  still  be  investigated  with  a  more 
robust  instrument  in  the  future. 
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Hypothesis  4 


IT  procurements  are  not  perceived  as  contributors  to  work  center 
productivity. 

Interpretation 

Both  groups  feel  IT  related  expenditures  have  improved  or 
enhanced  the  way  their  customers  complete  their  mission,  but  have  not 
allowed  the  organizations  to  reallocate  resources  to  expand  or  diversify 
their  missions.  The  first  factor  indicates  that  there  is  a  perception  that  IT 
procurements  do  indeed  positively  impact  work  center  productivity.  The 
second  factor  indicates  that  these  positive  impacts  to  work  center 
productivity  are  limited  to  the  scope  of  their  existing  mission,  thus  failing 
to  allow  for  future  mission  expansion. 

Implications 

Both  groups  have  high  alphas,  but  the  reallocation  of  resources 
factor  has  an  indication  that  a  possible  bias  exists  due  to  unequal  variances 
between  the  groups.  Therefore,  caution  should  be  used  when  making  a 
generalization  to  the  entire  population  even  though  the  Cronbach’s  alpha 
for  each  factor  is  high.  The  results  indicate  that  the  personnel  at  the  work 
centers  do  have  a  perception  that  IT  does  positively  impact  productivity. 
This  impact  is  limited  but  the  perception  is  still  a  positive  one,  thus 
Hypothesis  4  is  not  supported. 
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Hypothesis  5 

The  senior  leadership  at  the  work  center  level  has  a  significantly 
different  level  of  understanding  technology  turnover  and  its  effects  than  their 
network  maintainers. 

Interpretation 

The  two  groups  were  statistically  equivalent  on  eight  of  the  13 
factors  that  are  part  of  this  model.  The  five  factors  where  there  was  a 
statistically  significant  difference  are  procurement  process  changes, 
availability  of  funds,  internal  forces,  external  forces,  and  mission 
completion  enhancement.  However,  three  of  these  five  factors  showed 
implications  that  unequal  variances  could  have  contributed  to  the 
statistical  difference  between  the  mean  responses.  Therefore,  it  is 
difficult  to  make  inferences  on  the  reasons  for  the  difference  in  means  of 
these  three  factors.  As  for  the  remaining  two  factors,  these  differences 
indicate  that  there  are  some  opposing  ideologies  between  the  commanders 
and  their  maintainers  on  some  critical  issues  in  the  IT  arena. 

Implications 

The  results  above  show  a  partial  support  for  Hypothesis  5.  There 
are  five  factors  that  indicate  a  statistically  significant  difference  in  the  two 
groups’  responses.  The  first  of  these  factors  (procurement  process 
changes)  indicate  that  the  commanders  have  a  higher  understanding  of  the 
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procurement  process  changes  that  were  made  to  cope  with  the  perception 
of  an  IT  paradox.  This  may  be  due  to  poor  communications  within  the 
organizations  or  specific  need  to  know  restrictions,  but  as  mentioned 
above  the  unequal  variances  between  the  mean  responses  could  be  a 
contributing  factor  to  this  statistical  difference. 

The  next  factor  is  the  most  troubling  of  the  results  as  it  indicates 
that  the  commanders  feel  that  IT  funding  is  adequate  whereas  the 
maintainers  feel  that  it  is  not.  As  explained  in  the  analysis  of  Hypothesis 
3b  above,  this  disconnect  indicates  potential  problems  with  acquisition  of 
new  IT  systems.  However,  due  to  both  the  commanders  and  maintainers 
mean  responses  being  close  to  the  median  of  four  for  this  instrument,  it  is 
suggested  that  this  potential  problem  be  studied  in  future  research. 

The  next  two  factors  deal  with  the  environmental  influence  on  IT 
planning.  In  both  cases,  the  commanders  have  a  greater  perception  of  the 
internal  and  external  forces  that  impact  IT  spending.  This  makes  logical 
sense  as  they  in  fact  are  in  charge  of  the  organization  and  must  interact 
with  all  forces  regularly  to  complete  the  mission.  The  difference  between 
these  two  groups  may  be  explained  by  the  location  proximity  and  the 
necessarily  narrowly  focused  support  requirements  of  the  maintainers  but 
as  mentioned  previously  the  unequal  variances  between  the  mean 
responses  could  be  a  contributing  factor  to  this  statistical  difference. 

The  last  factor  is  also  troubling  as  it  indicates  that  the  personnel 
most  closely  tied  to  the  operation  of  IT  equipment,  the  maintainers,  have 
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a  lower  perception  of  IT’s  value  towards  work  center  productivity  than  do 
the  commanders  in  charge  of  procuring  it.  This  means  that  commanders 
could  have  a  higher  expectation  of  productivity  gains  for  an  IT  purchase 
than  is  feasible,  thus  potentially  supporting  a  further  perception  of  a 
paradox. 

Overall  Assessment 

The  research  indicates  that  there  may  be  a  perception  of  an  IT  productivity 
paradox,  even  though  Hypothesis  1  was  not  supported.  Both  commanders  and 
maintainers  feel  that  procurement  and  administrative  changes  have  been  made  in  IT 
planning  due  to  the  understanding  of  a  potential  IT  productivity  paradox.  The  apparent 
disconnect  between  the  results  of  Hypotheses  1  and  2  should  be  studied  in  greater  detail, 
but  could  possibly  be  explained  by  the  low  reliability  of  the  survey  instrument  used  to 
test  Hypothesis  1 .  The  important  thing  to  note  is  that  IT  planning  changes  have  been 
made  to  account  for  the  recent  technology  life  cycle  reductions  prevalent  in  the 
information  age.  Whether  it  is  called  the  IT  productivity  paradox  or  not,  the  results  are 
still  the  same,  Air  Force  work  centers  are  adapting  their  IT  procurement  strategies  to 
account  for  this  change  in  the  technological  marketplace. 

Ultimately,  the  Air  Force  work  centers  have  the  perception  that  they  are  getting 
and  adequate  return  on  investment  for  IT  expenditures,  indicating  that  their  IT  planning 
procedures  have  been  effective.  However,  the  results  also  indicate  that  they  have  a 
perceived  need  for  newer  technologies  to  be  able  to  keep  their  network  infrastructures  to 
the  necessary  level  to  support  their  customer’s  needs.  This  indicates  that  IT  planning  in 
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the  Air  Force  must  continually  change  to  strike  the  appropriate  balance  between  the 
demands  of  the  customers  and  the  capabilities  of  the  technologies. 

Limitations  of  Research 

This  research  has  many  limitations  that  constrain  the  certainty  of  the  results.  The 
single  most  limiting  factor  was  the  survey  itself.  It  was  constructed  using  a  series  of 
newly  fashioned  questions  that  were  thought  to  be  representative  of  how  perceptions  of 
each  of  the  phenomena  to  be  tested  were  formed.  This  was  necessary  because  a  single 
preexisting  survey  was  not  found  to  address  all  of  the  factors  and  hypotheses  needed. 
Therefore,  instead  of  constructing  a  new  survey,  a  more  thorough  search  of  existing 
surveys  should  be  done  and  a  piece  mill  assessment  survey  should  be  created  from  these 
previously  validated  surveys  to  allow  for  a  more  rigorous  and  accurate  assessment  of  the 
theories  being  tested.  A  further  limitation  of  this  type  of  survey  is  the  self  reporting  of 
the  respondents’  perceptions.  The  survey  answers  could  be  skewed  by  the  respondent  to 
make  the  outcomes  appear  as  they  feel  they  should  rather  than  their  actual  assessment  of 
the  phenomena  being  tested.  Considering  these  limitations,  the  results  of  this  research 
are  still  valid,  but  some  areas  could  not  be  accurately  tested.  Further  research  should  be 
done  in  these  areas  to  allow  for  a  more  thorough  understanding  of  the  IT  productivity 
phenomena. 

Another  limitation  was  identification  and  access  to  the  population  being  surveyed. 
Physical  access  was  eased  through  the  web-based  posting  of  the  survey  and  email 
notification  of  location  and  survey  response  times.  However,  the  Air  Force  is  moving 
towards  privatization  of  IT  and  computer  networking  functions.  This  meant  that  access 
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to  the  people  in  charge  of  networks  could  be  through  civilian  or  contractor  rather  than 
strictly  military  channels.  Since  the  Air  Force  has  not  entirely  migrated  to  this 
privatization  concept,  the  target  population  was  mixed  between  military,  civilian,  and 
contractor  personnel.  This  revelation  was  not  understood  until  the  pilot  study  was 
completed.  The  original  intent  was  to  assess  military  personnel’s  perceptions  only,  as  the 
assumption  at  the  start  of  the  research  was  that  control  would  still  be  performed  by 
military  personnel  even  if  IT  privatization  had  taken  place  at  particular  communication 
squadrons.  As  such  the  survey  was  only  approved  for  dissemination  to  military 
personnel.  To  cope  with  the  restrictions  made  evident  by  the  pilot  study,  the  survey  was 
sent  to  all  the  communication  squadron  commanders  with  the  caution  that  the  survey 
could  only  be  given  to  civilians  or  contractors  on  a  voluntary  basis  only.  This  could  have 
contributed  to  the  relatively  low  response  rate  of  the  survey.  To  cope  with  these 
restrictions  a  more  detailed  survey  approval  process  would  have  been  needed  to  address 
the  potential  civilian  and  contractor  respondents.  The  decision  was  made  to  proceed  with 
the  approved  survey  as  this  more  detailed  process  would  have  extended  the  timeline  of 
research  to  a  point  that  would  have  made  completion  under  the  current  time  restrictions 
infeasible.  Results  indicated  that  civilian  and  contractor  personnel  still  responded  to  the 
survey  on  a  voluntary  basis,  but  the  commanders  were  limited  in  their  abilities  to 
encourage  these  two  groups  to  participate  on  a  larger  scale. 

With  the  understanding  of  a  more  diverse  population  of  respondents  comes  a 
realization  that  access  to  the  respondents  could  be  difficult.  A  list  of  communication 
squadron  commanders  was  received  form  AFPC,  but  a  list  of  all  potential  respondents  to 
include  civilian  and  contractor  personnel  could  be  impossible  because  of  this  diversity 
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and  the  continual  push  towards  privatization.  It  is  recommended  that  if  this  type  of 
research  is  performed  again  for  this  population  that  this  aspect  be  addressed  with  AFPC 
personnel  before  survey  construction  takes  place  to  determine  if  contact  of  all  relevant 
personnel  is  feasible. 

A  further  limitation  could  have  been  the  willingness  a  commanders  to  participate 
in  this  survey  as  an  organization.  Due  to  the  diverse  nature  of  the  enlisted  network 
maintenance  field,  contact  to  the  network  maintainers  was  left  entirely  up  to  the 
commanders  who  received  the  survey.  This  could  introduce  a  bias  to  the  results  as 
commanders  could  have  selected  only  those  maintainers  that  were  ideologically  aligned 
with  themselves,  or  chose  not  to  include  any  of  their  maintainers  so  as  not  to  increase 
their  already  overburdened  workload. 

Another  limitation  is  the  assumption  of  independence  between  the  hypotheses 
being  tested.  It  is  easily  seen  that  one  hypothesis  could  have  a  correlation  to  the  others. 
The  research  undertaken  in  this  study  should  be  divided  in  to  the  distinct  groups  and 
performed  in  a  more  rigorous  manner  before  they  are  looked  at  in  totality  so  causal 
relationships  can  be  made. 

Future  Research 

Each  of  the  hypotheses  in  this  research  should  be  looked  at  in  individual  studies. 
The  results  indicate  that  there  are  phenomena  in  each  of  the  hypotheses  that  warrant 
future  investigation.  As  stated  in  the  limitations  above,  the  instrument  used  to  measure 
these  hypotheses  lacked  validity  in  certain  areas  such  as  customer  satisfaction  and 
technology  turnover’s  direct  impact  to  IT  planning. 
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Additionally,  an  area  was  identified  in  the  environmental  influences  hypothesis 
that  indicated  that  internal  and  external  forces  are  conflicting  when  it  comes  to  workplace 
IT  planning.  The  degree  and  interrelation  of  these  forces  should  be  studied  in  detail  to 
determine  what  impact  they  truly  have  to  workplace  IT  planning. 

Another  area  of  future  study  should  be  in  investigating  the  perceptions  of 
available  and  sufficient  funding  for  IT  expenditures.  The  results  indicated  that  the 
commanders  and  maintainers  have  significantly  different  perceptions  of  these  funding 
issues  and  this  could  be  a  critical  conflict  when  it  comes  to  planning  IT  expenditures  and 
assessments  of  current  system  viability. 

Since  this  study  was  a  point  in  time  assessment  of  the  phenomena  being  tested,  it 
is  suggested  that  this  research  be  repeated  periodically  to  produce  a  time  series  analysis 
of  the  data.  It  has  been  postulated  that  the  influence  on  workplace  productivity  from  IT 
parallels  the  productivity  impacts  realized  by  the  introduction  of  the  electric  motor  in  the 
Industrial  Revolution  (David,  1990).  An  assessment  in  a  time  series  formatted  study 
could  revel  that  as  IT  becomes  more  ingrained  in  the  way  we  operate,  as  did  the  electric 
motor,  we  will  see  productivity  gains  associated  with  procurements. 

Lastly,  the  results  of  this  study  indicate  that  a  significant  difference  is  evident 
between  the  perceptions  of  commanders  and  maintainers  with  respect  to  IT’s  influence  on 
workplace  productivity.  The  difference  indicates  that  maintainers  have  lower  perception 
of  IT’s  influence  on  productivity  than  do  their  commanders.  This  could  lead  to  a  problem 
in  assessing  return  on  investment  for  IT  expenditures  and  evaluating  the  need  for  future 
system  upgrades.  A  more  detailed  study  should  be  performed  to  investigate  this 
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difference  and  see  if  it  contributes  to  a  greater  perception  of  IT  productivity  paradox  in 
one  group  versus  the  other. 
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Appendix  A 


The  following  are  histograms  of  each  of  the  factors  relating  to  their  associated 
hypotheses  tests. 


Figure  1:  Hypothesis  1  Return  on  Investment  factor  histogram  (Commanders) 


Figure  2:  Hypothesis  1  Return  on  Investment  factor  histogram  (Maintainers) 
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Figure  3:  Hypothesis  1  Technology  Advancements  factor  histogram  (Commanders) 


Figure  4:  Hypothesis  1  Technology  Advancements  factor  histogram  (Maintainers) 


64 


H2_Process_changes_Commanders-  Distribution 


JSJx] 


’r  Distributions 


’*•  H2 Process changes Commanders 


0 

1 _ 1 

— 

i-  p  p  n  ] 

p 

h  [— i p-i  f-1-!  p-i 

1  2  3  4  5  6 

1  r  1 

7 

Quantiles 


Moments 


100.0%  maximum 

7.0000 

Mean 

4.9571429 

99.5% 

7.0000 

Std  Dev 

1 .5782831 

97.5% 

7.0000 

Std  Err  Mean 

0.1 089119 

90.0% 

6.0000 

upper  95%  Mean 

5.1718495 

75.0% 

quartile 

6.0000 

lower  95%  Mean 

4.7424362 

50.0% 

median 

6.0000 

N 

210 

25.0% 

quartile 

4.0000 

10.0% 

2.0000 

2.5% 

2.0000 

0.5% 

1 .0000 

0.0% 

minimum 

1 .0000 

J 

Figure  5:  Hypothesis  2  Procurement  Process  Changes  factor  histogram  (Commanders) 


Figure  6:  Hypothesis  2  Procurement  Process  Changes  factor  histogram  (Maintainers) 
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Figure  7:  Hypothesis  2  Administrative  Changes  factor  histogram  (Commanders) 


Figure  8:  Hypothesis  2  Administrative  Changes  factor  histogram  (Maintainers) 


66 


Figure  9:  Hypothesis  3a  Technology  Availability  factor  histogram  (Commanders) 


Figure  10:  Hypothesis  3a  Technology  Availability  factor  histogram  (Maintainers) 
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Figure  11:  Hypothesis  3a  Perceived  Technology  Need  factor  histogram  (Commanders) 


Figure  12:  Hypothesis  3a  Perceived  Technology  Need  factor  histogram  (Maintainers) 
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Figure  13:  Hypothesis  3b  Budgeted  Funding  factor  histogram  (Commanders) 


Figure  14:  Hypothesis  3b  Budgeted  Funding  factor  histogram  (Maintainers) 
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Figure  15:  Hypothesis  3b  Fallout  Funding  factor  histogram  (Commanders) 


Figure  16:  Hypothesis  3b  Fallout  Funding  factor  histogram  (Maintainers) 
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Figure  17:  Hypothesis  3c  Internal  Forces  factor  histogram  (Commanders) 


Figure  18:  Hypothesis  3c  Internal  Forces  factor  histogram  (Maintainers) 
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Figure  19:  Hypothesis  3c  External  Forces  factor  histogram  (Commanders) 


Figure  20:  Hypothesis  3c  External  Forces  factor  histogram  (Maintainers) 
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Figure  21:  Hypothesis  3d  Customer  Satisfaction  factor  histogram  (Commanders) 


Figure  22:  Hypothesis  3d  Customer  Satisfaction  factor  histogram  (Maintainers) 
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Figure  23:  Hypothesis  4  Mission  Enhancement  factor  histogram  (Commanders) 


Figure  24:  Hypothesis  4  Mission  Enhancement  factor  histogram  (Maintainers) 
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Figure  25:  Hypothesis  4  Reallocation  of  Resources  factor  histogram  (Commanders) 


Figure  26:  Hypothesis  4  Reallocation  of  Resources  factor  histogram  (Maintainers) 
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Appendix  B 


The  following  are  analyses  of  variances  (ANOVA)  for  the  responses  between  the 
two  groups  for  the  factors  relating  to  the  hypotheses  tested.  This  is  the  data  used  in  the 
assessment  of  Hypothesis  5. 


Figure  1:  ANOVA  for  Return  on  Investment  factor  of  Hypothesis  1 
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Figure  2:  ANOVA  for  Technology  Advancements  factor  of  Hypothesis  1 
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Figure  3:  ANOVA  for  Procurement  Process  Changes  factor  of  Hypothesis  2 
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Figure  4:  ANOVA  for  Administrative  Changes  factor  of  Hypothesis  2 
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Figure  5:  ANOVA  for  Technology  Availability  factor  of  Hypothesis  3a 
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Figure  6:  ANOVA  for  Perceived  Technology  Need  factor  of  Hypothesis  3a 
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Figure  7:  ANOVA  for  Budgeted  Funding  factor  of  Hypothesis  3b 
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Figure  8:  ANOVA  for  Fallout  Funding  factor  of  Flypothesis  3b 
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Figure  9:  ANOVA  for  Internal  Forces  factor  of  Hypothesis  3c 
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Figure  10:  ANOVA  for  External  Forces  factor  of  Flypothesis  3c 
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Figure  11:  ANOVA  for  Customer  Satisfaction  factor  of  Hypothesis  3d 
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Figure  12:  ANOVA  for  Mission  Enhancement  factor  of  Hypothesis  4 
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Figure  13:  ANOVA  for  reallocation  of  Resources  factor  of  Hypothesis  4 
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Appendix  C 


Figure  1:  Test  of  unequal  variances  Return  on  Investment  factor  Hypothesis  1 
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Figure  2:  Test  of  unequal  variances  Technology  Advancements  factor  Hypothesis  1 
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Figure  3:  Test  of  unequal  variances  Procurement  Process  Changes  factor  Hypothesis  2 
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Figure  4:  Test  of  unequal  variances  Administrative  Changes  factor  Hypothesis  2 
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Figure  5:  Test  of  unequal  variances  Technology  Availability  factor  Hypothesis  3a 
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Figure  6:  Test  of  unequal  variances  Perceived  Technology  Need  factor  Hypothesis  3a 
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Figure  7:  Test  of  unequal  variances  Budgeted  Funding  factor  Hypothesis  3b 
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Figure  8:  Test  of  unequal  variances  Fallout  Funding  factor  Flypothesis  3b 
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Figure  9:  Test  of  unequal  variances  Internal  Forces  factor  Hypothesis  3c 
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Figure  10:  Test  of  unequal  variances  External  Forces  factor  Hypothesis  3c 
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Figure  11:  Test  of  unequal  variances  Customer  Satisfaction  factor  Hypothesis  3d 
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Figure  12:  Test  of  unequal  variances  Mission  Enhancement  factor  Hypothesis  4 
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Figure  13:  Test  of  unequal  variances  Reallocation  of  Resources  factor  Hypothesis  4 
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